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Abstract 

We study a class of optimal control problems with state constraints where the state 
equation is a differential equation with delays. This class includes some problems arising in 
economics, in particular the so-called models with time to build, see [UISlEl]- We embed the 
problem in a suitable Hilbert space H and consider the associated Hamilton-Jacobi-Bellman 
(HJB) equation. This kind of infinite-dimensional HJB equation has not been previously 
studied and is difficult due to the presence of state constraints and the lack of smoothing 
properties of the state equation. Our main result on the regularity of solutions to such a 
HJB equation seems to be completely new. More precisely we prove that the value function 
is continuous in a sufficiently big open set of H, that it solves in the viscosity sense the 
associated HJB equation and it has continuous classical derivative in the direction of the 
"present" . This regularity result is the starting point to define a feedback map in classical 
sense, which gives rise to a candidate optimal feedback strategy for the problem. The study 
of verification theorems and of the closed loop equation will be the subject of the forthcoming 
paper pp] . 
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1 Introduction 

The main purpose of this paper is to prove a regularity result for a class of first order infinite 
dimensional HJB equations associated to the optimal control of deterministic delay equations 
arising in economic models. 

The regularity of solutions of the HJB equations arising in deterministic optimal control 
theory is a crucial issue to solve in a satisfactory way the control problems. Indeed, even in 
finite dimension, in order to obtain the optimal strategies in feedback form one usually needs the 
existence of an appropriately defined gradient of the solution. It is possible to prove verification 
theorems and representation of optimal feedbacks in the framework of viscosity solutions, even if 
the gradient is not defined in classical sense (see e.g. [HI [28]), but this is usually not satisfactory 
in applied problems since the closed loop equation becomes very hard to treat in such cases. 

The need of regularity results for HJB equations is particularly important in infinite 
dimension since in this case verification theorems in the framework of viscosity solutions are 
rather weak and in any case not applicable to problems with state constraints (see e.g [17^ I26j). 
To the best of our knowledge regularity results for first order HJB equation have been proved 
by method of convex regularization introduced by Barbu and Da Prato [3] and then developed 
by various authors (see e.g. [Il[5l[6l[7l[lll[l5l[l8l[2ll[22]). All these resuhs do not hold in the 
case of state constraints and, even without state constraints, do not cover problems where the 
state equation is a nonlinear differential equation with delays. In the papers [101 [TTl [T9] a class 
of state constraints problems is treated using the method of convex regularization but the 
type regularity is not proved. 

In this paper we deal with a class of optimal control problems where, given a control c > 
the state x satisfies the following delay equation 



x'it) = rxit) + fo (^x{t), J^^aiOxit + OdC) - cit) 
x(0) = r]o, x{s) = r]i{s), s G [-T,0), 



with state constraint x{-) > 0. The objective is to maximize the functional 




> 
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over the set of the admissible controls c. We will give a more precise formulation of the problem 
and of the hypotheses on the data in the next section. For the moment we observe that this 
kind of problems arises in various economic models (see e.g. [HO [25]), where the authors study 
optimal growth in presence of time-to-build (i.e. delay in the production due to the need of time 
to build new products) and cannot be treated with the existing theory except for very special 
cases (see the three papers just quoted). 

Using a standard approach (see e.g. [9]) we embed the problem in an infinite dimensional 
control problem in the Hilbert space H = x L^([— T, 0]; M), where intuitively speaking M 
describes the "present" and L^([— T, 0];M) describes the "past" of the system. The associated 
Hamilton-Jacobi-Bellman (HJB) equation in H has not been previously studied and is difficult 
due to the presence of state constraints and the lack of smoothing properties of the state equation. 

We prove that the value function is continuous in a sufficiently big open set of H (Proposition 
I3.10p . that it solves in the viscosity sense the associated HJB equation (Theorem 14. 3p and 
it has continuous classical derivative in the direction of the "present" (Theorem 14. 5p . This 
regularity result is enough to define the formal optimal feedback strategy in classical sense, 
since the objective functional only depends on the "present". The method we use to prove 
regularity is completely different from the one of convex regularization mentioned above. Indeed, 
it is based on a finite dimensional result of Cannarsa and Soner [12] (see also [8], pag. 80) 
that exploits the concavity of the data and the strict convexity of the Hamiltonian to prove 
the continuous differentiability of the viscosity solution of the HJB equation. Generalizing to 
the infinite dimensional case such result is not trivial as the definition of viscosity solution in 
this case strongly depends (via the unbounded differential operator A contained in the state 
equation) on the structure of the problem. In particular we need to establish specific properties 
of superdifferential that are given in Subsection 13.31 

We believe that such a method could be also used to analyze other problems featuring 
concavity of the data and strict convexity of the Hamiltonian. 

We finally observe that, even with our regularity result at hand the Verification Theorems 
and the study of the Closed Loop Equation associated to the problem are not trivial at all and 
will be the subject of the forthcoming paper |20| . 

The plan of the paper is as follows. Section [2] is devoted to set up the problem in DDE 
form giving main assumptions and some preliminary results (in Subsection 12. ip that are proved 
directly without using the infinite dimensional setting. In Section 3 we rewrite the problem 
in the infinite dimensional setting and prove existence and uniqueness of solutions of the state 
equation (Subsection l3.ip . continuity of the value function (Subsection l3.2p and useful properties 
of superdifferentials (Subsection 13. 3p . In Section 4 we apply the dynamic programming in the 
infinite dimensional context to get our main results: we prove that the value function is a 
viscosity solution of the HJB equation (Subsection 14. ip and then we prove a regularity result 
for viscosity solutions of HJB (Subsection 14. 2p . 

2 Setup of the control problem and preliminary results 

In this section we will formally define the control delay problem giving some possible applications 
of it. We will use the notations 

■= L^{[-T,0];R), and W]_'^ := W^'^{[-T,0];R). 
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We will denote by H the Hilbert space 
endowed with the inner product 

(•,•) = (•,-)r + (-,-)l2_, 

and the norm 



|2 I |2 2 



+ 

We will denote by r] := (770, Vii')) the generic element of this space. For convenience we set also 

H+ := (0,+oo) X Llj., H++ := (0, +00) x {r]i{-) € | r?i(-) > a.e.}. 

Remark 2.1. Economic motivations we are mainly interested in (see [Hill [25] and Remark 12.71 
above) require to study the optimal control problem with the initial condition in However, 
the set -f^4-+ is not convenient to work with, since its interior with respect to the || • ||-norm is 
empty. That is why we enlarge the problem and allow the initial state belonging to the class 
F+. ■ 

For r] G we consider an optimal control of the following differential delay equation: 
ix'{t) = rx{t) + /o (x{t), a{Ox{t + i)di) - c{t), 

\x{o) = m, x{s)=m{s), se[-T,o), 



(1) 



with state constraint x(-) > and control constraint c(-) > 0. We set up the following assump- 
tions on the functions a, /q. 

Hypothesis 2.2. 

• a(-) € W^'l^ is such that a(-) > and a{—T) = 0; 

• /o : [0, 00) X M ^ M is jointly concave, nondecreasing with respect to the second variable, 
Lipschitz continuous with Lipschitz constant Cj^, and 

/o(0,y)>0, Vy>0. (2) 



Remark 2.3. In the papers [H [21 [2S] the pointwise delay is used. We cannot treat exactly 
this case for technical reasons that are explained in Remark 14.81 below. However we have the 
freedom of choosing the function a in a wide class and this allows to take account of various 
economic phenomena. Moreover we can approximate the pointwise delay with suitable sequence 
of functions {a„} getting convergence of the value function and constructing e-optimal strategies 
(this approximation procedure is the object of the forthcoming paper 



From now on we will assume that /o is extended to a Lipschitz continuous map on setting 

fo{x,y) := fo{0,y), for x < 0. 

For technical reasons, which will be clear in Subsection 13. 2|, we work with the case r > 0, noting 
that nevertheless the case r < can be treated by shifting the linear part of the state equation. 
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Indeed in this case we can rewrite the state equation taking for example as new coefficient for 
the hnear part f = 1 and shifting the nonhnear term defining /o(x,y) = fo{x,y) — (1 — r)x. 

We say that a function x : [— T, oo) — > IR"*" is a solution to equation ([T]) if x{t) = r]i{t) for 
t £ [-T, 0) and 

x{t)=r]o + J rx{s)ds + j fo(^x{s),j a{^)x{s + ^)d^^ ds - J c{s)ds, t>0. 

Theorem 2.4. For any given r] € H^, c(-) € L^^^^([0, +oo); R"'"), equation ([T]) admits a unique 
solution that is absolutely continuous on [0, +oo) . 

Proof. Let K = sup^gj.j^o] For any t > 0, z^,z'^ £ C{[-T,t];R), we have 



r\zi{s) - Z2{s)\ + 



< 



r\zi{s) -Z2(s)| + C/o 



fo{zi{s), l\{Ozi{s + 0^ -fo(z2{s),l\{C)z2{s + 

zi{s) - Z2{s)\ + K r \z,{s + - Z2{s + 0\d^ 

J-T 



ds 



ds 



< 



(r + Cf,)\z,{s) - Z2{s)\ + Cf,K / \ziiO - ^2(01^? 



ds 



<ir + Cf,) f \zi{s) - Z2{s)\ds + tCf.K f \z^{i) - Z2mdi 

Jo J-T 

<[{r + Cf,) + tCf,K] f W{i)-Z2{im 

J~T 

< [{r + Cf,) + tCf,K]{t + T)i/2 |^,(^) _ Z2{0fd^^ 

Thus the claim follows by Theorem 3.2, pag. 246, of [9]. 
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□ 



We denote by x{-;r],c{-)) the unique solution of ([T|) with initial point rj € if+ and under 
the control c(-). We emphasize that this solution actually satisfies pointwise only the integral 
equation associated with ([T]); it satisfies ^ in differential form only for almost every t £ [0, +oo). 

For r] E we define the class of the admissible controls starting from r] as 
Civ) := {c(-) GLL([0,+oo);M+) | x{-,rj,c{-)) > 0}. 

Setting x{-) := x{-, ;r/,c(-)), the problem consists in maximizing the functional 

r+oo 

J(r?; c(-)) := / e""* [C/i(c(t)) + ^72(x(t))] dt, p > 0, 
Jo 

over the set of the admissible strategies. 

The following will be standing assumptions on the utility functions Ui, U2, holding through- 
out the whole paper. 



Hypothesis 2.5. 
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(i) Ui E C([0,+cx));M) n C2((0, +00); M), U{ > 0, [/((0+) = +00, < and Ui is bounded. 

(ii) U2 S C((0, +oo);R) is increasing, concave, bounded from above. Moreover 

r+00 

/ e-P^U2 (e-^/o*) dt > -00. (3) 
Jo g 

Since Ui, U2 are bounded from above, the previous functional is well-defined for any rj € 
and c(-) G C{i]). We set 

Ui := lim Ui{s), U2 := Hm C/2(s). 

Remark 2.6. We give some comments on Hypothesis 12.51 and on the structure of the utility in 
the objective functional. 

1. Through the whole paper the case C/2 = is allowed. Therefore, the case of an objective 
functional depending only on consumption (as in [Il[2l[25]) is allowed. 

2. The assumption that Ui, U2 are bounded from above is done for simplicity to avoid too 
many technicalities. It guarantees that the value function is bounded from above and this 
fact simplifies arguments in the following parts of this paper. Similarly the assumption 
that Ui is bounded from below guarantees that the value function is bounded from below. 
We think that it is possible to replace such assumptions with more general conditions 
relating the growth of Ui, U2, the value of p and the parameters of the state equation. 
Typically, such a condition requires that p is sufficiently large. 

3. All utility functions bounded from below satisfy ([3]). Also U2{x) = log(x), U2{x) = 
x"^ , ^ > —-TT-, satisfy ([3]). Note also that (13|) is equivalent to 

r+00 

/ e-P^U2 (Ce"^^o*) dt > -00, > 0. 
Jo 

4. When r < 0, then in ([3]) we have to replace Cj^, with |r| + Cj^,. 

5. If we assume that 

35 >0 such that rx + /o(x,0) > 0,Vx G (0,<5], (4) 

then the assumption ([3]) can be suppressed. Since ([2]) implies /o(0,0) > 0, (H]) occurs for 
example if x 1— > rx + /o(x,0) is nondecr easing, therefore in particular if r > and /o 
depends only on the second variable. 

6. We think that it should be possible to treat also the case of a utility function U depending 
on the couple (c, x); it should be enough to replace Hyphotesis 12.51 with the following 
assumptions: 

• U : [0, +00) X (0, +00) — > M is concave and increasing with respect to both the 
variables and bounded from above; 

• for any x > the function U{-,x) belongs to the class C([0, +00); M)nC^((0, +00); R), 
[/'(•, x) > 0, C/'(0+,x) = +00, [/"(•, x) < 0. 
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Remark 2.7. The control problem described above covers also the following optimal consump- 
tion problem. We may think of the dynamics defined in ([T]) as the dynamics of the bank account 
driven by a contract which takes into account the past history of the accumulation of wealth. 
Such a situation arises when the bank offers to the customer an interest rate r smaller than the 
market spot rate r*^ and as a compensation, it provides a premium on the past of the wealth 
(this may happen e.g. for pension funds). Then the following equation is a possible simple 
model of the evolution of the bank account under such a contract: 

x'{t) = rx{t) + go ( a{Ox{t + ^)dc) - c(t), 
x(0) =r?o, x(s) =r?i(s), se[-T,0), 

where (70 : M ^ M is a concave, Lipschitz continuous and strictly increasing function such that 
9o(0) > 0. Dependence on the past is an incentive for the customer to keep his investments with 
the bank for a longer period of time in order to receive gains produced by the term gQ. Here we 
assume the point of view of the customer and we think it is interesting to study the behaviour 
of the optimal consumption in this setting, comparing it with the one coming from the classical 
case, which corresponds to set r = r*^, g = 0. 

We think also that our technique could be adapted to cover optimal advertising model with 
nonlinear memory effects (see e.g. [23| on this subject in a stochastic framework). ■ 

For 1] G the value function of our problem is defined by 

V{r]):= sup J(r/,c(-)), (5) 

c{-)ec{v) 

with the convention sup0 = — 00. The domain of the value function is the set 

V{V) := {r? G I V{r]) > -00}. 
Due to the assumptions on Ui, U2 we directly get that V < ^{Ui + ^^2)- 

2.1 Preliminary results 

In this subsection we investigate some first qualitative properties of the delay state equation and 
of the value function. 

Lemma 2.8 (Comparison). Let rj G i7+ and let c(-) G L\^^{[Q,+oo);W^). Let x{t), t >0, be an 
absolutely continuous function satisfying almost everywhere the differential inequality 

(x'{t) < rx(t) + /o (x{t), a{Ox{t + ^d^) " c{t), 
[x{0) < rjo, x{s) < r/i(s), for a.e. s G [-T,0). 

Then x(-) < x(-; ry, c(-)). 



Proof. Set a := sup^gj.^-o] 1*^(01) vi') •= 3;(-; ry, c(-)) and /i(-) := [x{-) — y(-)]"^. We must 
show that h{-) =0. Let e > be fixed and such that eC f^aTe''''''^^fo^ < 1/2 and let M := 
maXigjo,e] h{t)- By monotonicity of /o we get 

/o (^x(t), y\(e)x(t + Od^^ < fo (^x{t), f^a{i)y{t + i)di + aTM^ , for t G [0,e]. (6) 
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Define, for n G N, 



for X < 0, 
for x G (0, l/2n] 



<fn{x) := < nx^, 

— l/4n, for x > l/2n. 

The sequence (v?n)neN C C-^(M;M) is such that 

Vnix) = (Pn{x) = 0, for every x G (— oo,0], n G N, 
< f'nix) < 1, for every x G M, n G N, 
'/'n(2^) — > a;"*", uniformly on x G M, 
ip'nix) 1, for X G (0,+oo). 

We can write for t G [0, e], taking into account ([6]), 

^n[x{t) - y{t)) = ^Pn{x{0) - rjo) + ip'^{x{s) - y{s))[x'is) - y'{s)]ds 



< 



[ Vnixis) - y{s)) 

Jo 



ds 



< I ip'^{x{s) - y{s)) 

fO 



r(x(s) - y{s)) 

+/o [x{s), a{Ox{s + - /o + O^^c) 

+/o + O^^e + ciTM^ - /o a{Oy{s + ^ 

< f ip'Ms)-y{s))\{r + Cf,)\x{s)-y{s)\ + CaTM]ds. 
Jo L J 

Letting n — > oo we get 

h{t) < [ {r + CfJh{s)ds + CfodTMt< [ {r + Cf^) h{s)ds + Cf^dTMe. 
Jo Jo 

Therefore by Gronwall's Lemma we get 

Ht) < eCf^dTMe^^''+^io\ for t G [0,e], 

and by definition of e 

M 

hit)<—, fortG[0,e]. 

This shows that M = 0, i.e. that /i = on [0, e]. Iterating the argument, since e is fixed, we get 
/i = on [0, +cxd), i.e. the claim. □ 

Proposition 2.9. We have 

H++ C ViV), V{V) = {r? G i/+ I G C{r,)}. 



ds 
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Proof. Let r] £ and set x(-) := x{-;r],0). By assumption, x{0) = rjo > and until 

x(t) > we have 

x'{t) = rx{t) + /o (^x{t), J ^ a{0<t + ^d^^ > rx{t) + /o(x(t), 0). 

Since /o(0, 0) > and /o(-,0) is Lipschitz continuous (with Lipschitz constant Cf^), we get 

x'{t) > -Cfo x(t), until x(t) > 0. 

This fact forces to be 

inf{t > I x{t) = 0} = +00, 

and x{t) > r]Qe~^fo^ for any t >0, which proves the inclusion C 'D{V) thanks to ([3]). 

Now let r] S T>{V); then, by definition of ViV), there exists c(-) € C{r]). By Lemma 12.81 
G C{ri), so that we have the inclusion "D^V) C {r/ E \ € C{r])}. Conversely let r] € be 
such that G C{r]). Then, by definition of C{r]), we have inf^gp^T] x(t;77, 0) > ^ > 0. Repeating 
the argument used above, we get x(t;ry,0) > ^e~'^^o^*~^^ for t > T, so that G and the 

proof is complete. □ 

Remark 2.10. It is straightforward to see that the proof of Proposition 12.91 above works if we 
replace the assumption ([3]) with the assumption (jl]). ■ 

Definition 2.11. (i) Let r/ G Vly). An admissible control c*(-) G C{rf) is said to be optimal 
for the initial state i] if J{r]\c*{-)) = V{Tj). In this case the corresponding state trajectory 
x*{-) := x{-;r],c* {■)) is said to be an optimal trajectory and the couple {x* {■) , c* {■)) is said an 
optimal couple. 

(ii) Let T] G '^{V), e > 0; an admissible control c^(-) G C{ri) is said e-optimal for the 
initial state rj if J{ri;c^{-)) > V{ri) — e. In this case the corresponding state trajectory x^(-) := 
x{-;r], {■)) is said an e-optimal trajectory and the couple {x'^ {•) , {■)) is said an e-optimal 
couple. □ 

Proposition 2.12. The set ViV) is convex and the value function V is concave on ViV). 

Proof. Let r?,r? G T>{V) and set, for A G [0,1], rjx = Xr] + {1 - X)fj. For e > 0, let 
c^(-) G C{r]), (f{-) G C(f/) be two controls e-optimal for the initial states r],fi respectively. 
Set x{-) = j;(-,r/,c=(-)), x(-) = x{-,r], c^(-) = Ac^(-) + (1 - A)c^(-). Finally set xa(-) = 

Xx{-) + (1 — X)x{-). Let us write the dynamics for xx{-)- 

x'x{t) = Xx'{t) + {1- X)x{t) 



A 



rx{t) + /o (^x(t), a{Ox{t + Od^^ - c'{t) 



+(1 - A) 



< rxA(t) + /o XA(t), 



f^a{i)xx{t + Oddj -c\t), 



where the inequality follows from the concavity of /o with initial condition ry^- Let r/^, c'^(-)) 
be a solution of the equation 



x'{t) = rx{t) + /o x(t) 



, f^a{i)x{t + Od^-c^{t). 
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Since xx{-) > by construction, by Lemma [2^ we have x{-;r]x,c^{-)) > xx{-) > 0. This shows 
that c^{-) € C{ri\). By concavity of f/i, U2 and by monotonicity of U2 we get 

V{vx) > J{w,c\-)) > XJ{v;c'{-)) + (1 - A)J(r?;c-(-)) > \V{v) + (1 - X)V{f]) - e. 
Since e is arbitrary, we get the claim. □ 

By assumptions of monotonicity of the utiUty functions and by Lemma 12.81 we obtain the 
fohowing result. 

Proposition 2.13. The function r] 1— > V{r]) is nondecreasing in the sense that 



□ 



Indeed, the value function is strictly increasing in the first variable: 
Proposition 2.14. We have the following statements: 
1- y{^) <-piPi + U2) for any r] H+. 

2. lim^o-+ool^(%,??i(-)) = -p{Ui + U2), for all r]i{-) e L^^. 

3. V is strictly increasing with respect to the first variable. 
Proof. 1. Let r] G ViV) and set 



a:= sup a(0, P ■= sup x{^;7j,0), q := r]f{^)d^. 

f6[-T,0] 5e[0,T] J-T 



Let c(-) € C{r]) and set x(-) := 77, c(-)); by comparison criterion we have x{t) < p in [0,T]. 

Since /o is Lipschitz continuous, there exists C > such that fo{x,y) < C(l + \x\ + \y\) for 
all x,y (zM. Therefore, for t E [0,T], we can write, considering the state equation in integral 
form, 

x{t) <rio + rTp + TC{l + \p\ + \a{Tp + q)\) - / c(r)dr. 

Jo 

CaU K := r]o + rTp + TC{l + \p\ + \a{Tp + q)\); since c(-) e C(r/), we have x{t) > in [0,r], so 
that ^ 

/ c{T)dT < K. 
Jo 

Denoting by m the Lebesgue measure, this means that 

m{r e [0, T] I c(r) < 2K/T} > T/2. 

Therefore (in the next inequality, since e~^^ is decreasing, we suppose without loss of generality 
that c(-) < 2K/T on [|,r]) 



/ e-P*Ui{c{t))dt < / e-P^Ui{c{t))dt+ / e-P^Ui{2K/T)dt + / e-''*C/i(c(t))(it 

JO Jt/2 Jt 

< ^ - r e-P^ ( Ui - Ui{2K/T)) dt. 
P Jt/2 ^ ' 
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Since the quantity Ui — Ui{2K/T) is strictly positive and does not depend on c(-), the claim is 
proved. 

2. For given ?7i(-) € L'^j^, let i^T > 0, Af > and let us define the control 



c(t) := 



0, if t > K. 



Take tjq > 0. Since /o is Lipschitz continuous and nondecreasing with respect to the second vari- 
able, we can see that, until it is positive, x{t; {r]o,r]i{-)),c{-)) satisfies the differential inequality 

(x'{t) > -C{l + x{t)+q)-M, 
|x(0) = 7]o, 



for some C > 0, where 



q:= ( sup a(C) ) ( T vHm) 

\£Gf-T,Ol / \J-T J 



This actually shows that, for any M > 0, A' > 0, > 0, we can find r\Q such that c(-) S 
C(r/o, and rc(-; (770, ?7i(-)), c(-)) > i? on [0,K]. By the arbitrariness of M,K,R the claim is 

proved. 

3. Fix 77i(-); we know that tjq is concave and increasing. If it is not strictly 

increasing, then it has to be constant on an half line [k, +00), but this contradicts the first two 
claims. □ 



3 The delay problem rephrased in infinite dimension 

Our aim is to apply the dynamic programming technique in order to solve the control problem 
described in the previous section. However, this approach requires a markovian setting. That 
is why we will reformulate the problem as an infinite-dimensional control problem. Let n = 
(1,0) G and let us consider, for i] G H and c(-) G L^([0, +0x0); the following evolution 
equation in the space H: 

(x'it) = AX{t)+F{X{t))-c{t)n, 
\X{0) =iieH+. 

In the equation above: 

• A : T>{A) C H — > H is an unbounded operator defined by A{riQ,rji{-)) := {rr]Q,7][{-)) on 

V{A) :={r]eH\ 771 (•) G W^^^, r?i(0) = %}; 

• F : H — > H is a Lipschitz continuous map defined by 

F(r?o,7?i(-)) := (/(r?o,r?i(-)),0), 
where /(r/o, r/i(-)) := /o imi fir "-(OmiOd-C ^ 
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It is well known that A is the infinitesimal generator of a strongly continuous semigroup {S{t))t>o 
on H; its explicit expression is given by 

S{t){r,o,Vi{-)) = {voe'\l[-T,o]{t + -) m{t + ■) + Ilo,+o.){t + ■) r?oe'^*+'^ 
About the estimate on the norm of the semigroup, we have 

\\S{t)vf < |%e'-f + 2 r |/[_r,o](i + C)^?i(i + C)|'dC 

J-T 

dc 



+2 [ I[o,+oc>){t + C) Voe 

J-T 

< ((3 + 2r))e2^'*||ry||2, 



rit+0 ^ 



I.e. 



\\Sit)\\c^H)<Me'^\ (8) 



where M = {3 + 2T), uj = 2r. 



3.1 Mild solutions of the state equation 

In this subsection we give a definition of the mild solution to ([7]), prove the existence and 
uniqueness of such a solution and the equivalence between the one dimensional delay problem 
and the infinite dimensional one. 

Definition 3.1. A mild solution of d?]) is a function X G C{[0,+oo); H) which satisfies the 
integral equation 



X{t) = S{t)r] + [ S{t- T)F{X{T))dT + [ c{T)S{t - T)h dr. 
Jo Jo 

Theorem 3.2. For any ij ^ H , there exists a unique mild solution of 



(9) 



Proof. Due to the Lipschitz continuity of F and to ([8]), the proof is the usual standard 
application of the fixed point theorem (see e.g. [9]). □ 

We denote by X{-;ri,c{-)) = {Xo{-;ri,c{-)),Xi{-;r],c{-))) the unique solution to ([7]) for the 
initial state r] ^ H and under the control c(-) G L^([0, +oo); M"*"). The following equivalence 
result justifies our approach. 

Proposition 3.3. Let r] G H^, c(-) G C{r]) and let x(-), X{-) be respectively the unique solution 
to ([ip and the unique mild solution to ^ starting from r] and under the control c(-). Then, for 
any t >0, we have the equality in H 

X{t) = {x{t),x{t + Oi^^^Tfl])- 

Proof. Let x(-) be a solution of ([I]) and let Z(-) := + C)lc6[-T,o])- Then Z(-) 

belongs to the space C([0, +oo); H) because the function [0, +oo) 9 t i— > x{t) G M is (absolutely) 
continuous. Therefore, it remains to prove that Z{t) = (Zo(t), Zi(t)) satisfies d?]) and then the 
claim will follow by uniqueness. For the first component we have to verify that, for any t > 0, 



Zo{t) = + ^*e^^*--)/o (zo(T),y\(e)Zi(r)(e)d^) dr - 



t 



c{T)dT, 
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i.e. that 



x{t) = + 1^ e^(*-^)/o (x(r), j''^ a(e)x(r + 0^^) dr - e^^'"-^ c{T)dT, 
but this follow from the assumption that x{-) is a solution to ([T]). 

For the second component, taking into account that /[o,+oo)(^ + • — t) = -/^[r,+oo)(^ + ")) '^^ have 
to verify, for any t > 0, for a.e. ( € [— T, 0], 

Zi (t) (C) = /[_T,o] (t + C)??i (t + C) + /[o,+oo) (t + C)%e''(*+«) 

JO 

i.e., for any t > 0, for a.e. C € [— T, 0], 

x(t + C) = /[-T,o] (i + C)??i (i + C) + /[o,+oo) {t + C) %e^(*+« 

+ ^ /[.,+oo)(t + e^(*+^-^)/o (x(r),y^a(0x(T + 0t^^) dr (10) 

- r/[.,+oo)(t + C) e^(*+f--)c(r)dT. 

JO 

For C G [-T, 0] such that t + C ^ [-T, 0], ^ reduces to 

2^(i + C) =m(i + C) 

and this is true since r]i is the initial condition of ([1]). If C ^ [— 0] is such that t + C>0, then 
([To]) reduces to 

x{t + C) = r/oe''(*+^) + y e''(*+^-"Vo y a(03;(r + dr - J e"(*+«-")c(r)dr. 

Setting M := t + C this equality becomes, for u > 0, 

x{u) = xoe™ + e^'("-^)/o (^x(r), a(02;(T + ()d^^ dr - e''^''~^^ c{T)dT. 
Again this is true because x(-) solves ([I]). □ 

3.2 Continuity of the value function 

In this subsection we prove a continuity property of the value function that will be useful to 
investigate the geometry of its superdifferential in the next subsection. To this end we recall 
that the generator A of the semigroup {S{t))t>o has bounded inverse in H given by 

(7?o,r/i) (s) = (7, y - /^i(^)'^^) ' ' € [-^'Oi- 



ls 



It is well known that is compact in H. It is also clear that is an isomorphism of H 
onto T>{A) endowed with the graph norm. 
We define the || • ||_i-norm on H by 

llr^ll-i := ||^~"^r/||. 

In the next proposition we characterize the adjoint operator A* and its domain 'D(A*). 

Proposition 3.4. Let rj = {'HOiVii')) ^ Then r] € D^A*) if and only if rji G W]_'^ and 
r]i{—T) = 0. Moreover, if this is the case, then 



Proof. Let 



Then, for C G T){A), 



A*7] = {ri]o + m{0),-r][{-))- 



{t]o,Vi) eV= \^r]eH : rjie W^'^, Vi{-T) = o} 
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(^C,^)=<0%+ / C,i{s)rii{s) ds = rQQ-qQ + C,Qrii{Q) 



Ci{sWi{s)ds, 



thus C i^CiV) is continuous on T>{A) with respect to the norm || • ||, i.e. rj G D{A*) and 

A*r] = {rr]o + miO),-v[i■))■ 
Theve^oTe, r] €V{A*) and (dH) holds. To show that V (A*) = V note first that for t < T 



S*{t) {vo,Vi{-)) = ( e^M r?o + / r?i(e)e^«dC ) ,r/i(- - t)/[_T,o](- " t) 



(12) 



Clearly, T> is dense in H and it is easy to check that S*{t)T> C "D for any t > 0. Hence by 
Theorem 1.9 on p. 8 of [13] V is dense in 'D{A*) endowed with the graph norm. Finally, using 
(jlip it is easy to show that P is closed in the graph norm of A* and we find that V (A*) = D. 
□ 

Lemma 3.5. The map F is Lipschitz continuous with respect to \\ ■ 

Proof. Due to the Lipschitz continuity of /o, it suffices to prove that 



l%l + 



T 



< C'a(-)lkll-i) Vr/ G -ff. 



(13) 



Indeed, since |?7o| < r||77||_i (0, a(-)) G T>{A*), we find that 



T 



aiOviim = |((0,a(-)),r/)| = |((0,a(-)),AA-S)l 

= |(A*(0,a(.)),A-S)|<P*(0,a(-))||-h||-i. 
So, since |?7o| < r||r/||_i, we get ([13]) with Ca(.) = r + ||A*(0, a(-))||. 



□ 
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Remark 3.6. The condition a{—T) = is in general necessary for the previous result. Indeed, 
consider for example the case a(-) = 1. Then the sequence 

< = := 0, <(•) := n>l, 

is such that ^ 

/ = 1 Vn > 1, ||r/"||_i ^ when n ^ cx), 

SO that (|13p cannot be satisfied. If for example, fo{r,u) = u, the previous result does not hold. 
■ 

Lemma 3.7. Let X{-), X(-) be the mild solutions to ^ starting respectively from r],f] & H and 
both under the null control. Then there exists a constant C > such that 

\\X{t)-X{t)\\.i<C\\r]-fi\\_i, yte[0,T]. 

In particular 

\Xo{t)-Xo{t)\<rC\\T]-fj\\^i, VtG [o,r]. 
Proof. From Q we can write, for all t G [0, T], 

X{t) - X{t) = -??)+/* S{t - r) [F{X{t)) - F(X(r))] dr, 

JO 

so that 

A~HX{t) - X{t)) = S{t)A-\r^ f S{t - t) A'' [F{X{t)) - F(X(t))] dr, 

Jo 

i.e., taking into account Lemma 13.51 there exists some K > such that 

\\X{t) - < K - ?7||_i + ^* \\X{t) - X{T)\UdT^ 

and the claim follows by Gronwall's Lemma. □ 



Proposition 3.8. The set T>[V) is open in the space {H, || • ||-i). 

Proof. Let fj € ViV), r] e H+ and set X(-) := X(-;r/,0), X{-) := X{-;r],0). By Proposition 

12.91 we have X{t) > ^ > for t € [0, T]. For any e G ^0, and any r/ such that ||r/ — r/||_i < e. 

Lemma [3.71 vields XQ{t) > ^/2 for t € [0, T]. Arguing as in Proposition 12.91 we get Xo^t) > 
i^-K{t-T) £qj. I yj" Thus we have the claim. □ 

Remark 3.9. Note that 'D{V) is open also with respect to || • ||. 

Proposition 3.10. The value function is continuous with respect to \\ ■ ||_i on T>(V). Moreover 
{Vn)cV{V), r,n^r^^V{V) =^ V {r^n) ^ V {rj) ■ (14) 

Proof. The function V is concave and, thanks to the proof of Lemma [3.8l it is || • ||_i-locally 
bounded from below at the points of T>{y). Therefore the first claim follows by a classic result 
of convex analysis (see e.g. [16], Chapter 1, Corollary 2.4). 

The claim p4p follows by the first claim and since A~^ is compact. □ 
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D-v{fi) := iCeH 



D+v{fi) ■.= \(eH 



3.3 Properties of superdifferential 

In this subsection we focus on the properties of the superdifferential of concave and || • ||_i- 
continuous functions. This will be very useful in proving a regularity result for the value function. 
Recall that, if f is a function defined on some open set O of H, the subdifferential and the 
superdifferential of u at a point f] € O are the convex and closed sets defined respectively by 

liminf^M^4^1^T^^>0 

v^v 11^ — ^11 

limsup^^M^4^Z)^7r^<0 

v^v \\v - v\\ 

It is well-known that, if D^v{r]) n D~v{rj) ^ 0, then D^v{rj) D D~v{r]) = {C}, v is differentiable 
at r] and Vv{ri) = Q. Moreover the set of the "reachable gradients" is defined as 

D*v{f]) := ^( G H Brjn fj such that 3Vv{7]n), Vv{7]n) c| • 

If O is convex and open and u : O — > M is concave, then the set D~^v is not empty at any point 
of O and 

D+v{f]) = ^C£H v{r]) -v{f]) < {r]-f],C), Vr] £ = cd {D*v{f])) . (15) 

Moreover in this case, if D^v{fj) = then v is differentiable at tj and Vv{r]) = 
Lemma 3.11. The following statements hold: 

1. A~'^{T>{y)) is a convex open set of {^{A), \\ ■ ||). 

2. O := ||.||) (C/os(j:^ ||.||-) (^A~^{'D{V)))) is a convex open set of {H, || • ||). 

3. Od A~\V{V)) and V{V) = O n V{A). 

Proof. The first and the second statement are obvious. We prove the third one. Of course, 
since A~^{'D{V)) is open in {V{A), \\ ■ ||), we can find i£x)xeA-'^{v{v))i > 0, such that 

A-\V{y))= IJ i?(^(A),||.||)(x,e,). 
x(^A-^{V{y)) 

By this representation of A~^{T>iy)) we can see that 

x&A~^(v{y)) 

Therefore we get both the claims of the third statement. □ 

Proposition 3.12. Let v : 'D(V) ^ M. be a concave function continuous with respect to \\ ■ 
Then 

1. V = u o A~^ , where u : O C H ^ is a concave \\ ■ \\- continuous function. 

2. D+v{rj) C P(A*), for any r] G T>{V). 
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3. D~^u{A ^T]) = A*D'^v{r]), for any i] € T>{V). In particular, since A* is injective, v is 
dijjerentiable at rj if and only if u is differentiable at A~^ri. 

4. If C ^ D*v{r]), then there exists a sequence ijn ^ rj such that there exist 'S/v{r]n), '^v{r]n) — s- 
C and A*Vv{'qn) ^ A*C. 

Proof. Within this proof, for i] € DiV), we set rj' := A'^rj. Since A~^ is one-to-one, there 
is a one-to-one correspondence between the elements r] G T^iV) and rj' E A~^{T>{V)). 

1. Let us define the function uq : A~^{T){V)) — > M by 

miv') ■= v{ri). 

Thanks to the assumptions on v, uq is a concave continuous function on {A^^(T>(V)), \\ ■ ||). By 
the third statement of Lemma [3.11l we see that A~^{T>{V)) is || • ||-dense in O. Since v is concave 
it is locally Lipschitz continuous, so that can be extended to a concave || ■ ||-continuous function 
u defined on O. This function u satisfies the claim by construction. 

2. Let f] G V{V), C G D+v{fi). Then 

v{r]) — v{'q) < {i] — fj, C), Vr/ G T>{V), 

i.e. 

u{7]') - u{fj') < {A{7]' - fj'),C), Vr?' G A~\V{V)). 

Thus the function 

T^: mA),\\-\\) M, 

rj' ^ {Ar,',0, 

is lower semicontinuous at fj'. It is also linear and therefore it is continuous on {'D{A), \\ ■ ||), so 
that we can conclude that G T>{A*). 

3. Let fj G V{V), C G D^v{f]). Then 

v{v) -v{v) < {f] -fi,C), \/rieV{V), 

i.e. 

u{v') - u{n') < {A{r,' - fj'), = {{r,' - fj'), A*C), Vr?' G A-\V{V)), 

so that A*( G D+u{fj'), which gives D'^u{A^^r]) D A*D+v{rj). 
Conversely let f]' £ A~^{T>{V)) and C' e D+u{fi'). Then 

n(r?') - u{f]') < (A(r?' - fj'), C), Vr/' G A-\V{V)), 

i.e. 

v{v) - v{f)) < {A-'ir, - fj), C) = {{rj - fj), {A~^)*C'), V?? G V{V). 

Since {A~^)* = (A*)'^, we get {A*)-^C' G D+v{fj), which gives D+u{A-^r]) C A*D+v{rj). 

4. Let fj G T^iV) and ( G D*v{fj). Due to (fTSi) . we can find a sequence rjn ^ fj such that 
^v{r]n) exists for any n G N and Vv{r]n) C- Thanks to the third claim we can say that also 
Vu{fj'^) exists and Vu{fj'^) = A*Vv{rin)- The sequence Vu{fj'^) is bounded, due to the fact that 
the set-valued map r]' 1— > D^u{fj') is locally bounded. Therefore from any subsequence we can 
extract a subsubsequence weakly converging to some element Q' G H. A* is a closed operator, 
so that it is also a weakly closed operator. Therefore we can conclude that C, G T)[A*) and 
Q' = A*C,. Since this holds for any subsequence, we can conclude that A*^v[rin) A*C,. □ 
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4 Dynamic Programming 



The dynamic programming principle states that, for any rj € 1^{V) and for any s > 0, 



V{r]) = sup 

c{.)6C(ry) 



' e-P' (UMt) + U2{Xom dt + e-P'V{X{s- r?, c(-))) 



Its differential version is the Hamilton- Jacobi-Bellman (from now on HJB) equation on T>{y), 
which in our case reads as 

pv{l^) = {rj,A*Vv{v)) + f{v)vr,,{v) + U2{m) + nvvoiri)), (16) 

where TC is the Legendre transform of Ui, i.e. 

7^(Co) :=sup(C/i(c) -Coc), Co>0. 

c>0 

Due to Hvphothesis I2.5l -(i) and to Corollary 26.4.1 of [27], we have that Tt is strictly convex on 
(0,+oo). Notice that, thanks to Proposition 12. 14T - (3) . 

K^ir]) ■■= {Co G M I (Co,Ci(-)) e D+V{v)} C (0, oo) 
for any i] G DlV), i.e. where Tl is defined. 

4.1 Viscosity solutions 

First we study the HJB equation using the viscosity solutions approach. In order to follow this 
approach, we have to define a suitable set of regular test functions. This is the set 

T:=[ipe C\H) I Vv9(-) G V{A*), r]n ^ ?? ^ A*\/ip{7]n) ^ A*V^{r])Y (17) 

Let us define, for c > 0, the operator on r by 

[C^VM := -p^ii]) + {7],A*Vipir])) + fiv)ipM - opnM- 

Lemma 4.1. LetipeT,c{-) G L^([0, +oo); M+) and set X{t) := X{t;r],c{-)). Then the following 
identity holds for any t > 0: 



e 



P'v{X{t))-^{v)= / e-P^[£<^^^]{X{s))ds. 



Proof. The statement holds if we replace A with the Yosida approximations. Then we can 
pass to the limit and get the claim thanks to the regularity properties of the functions belonging 

to T. □ 

Definition 4.2. (i) A continuous function v : 'D{V) ^ M is called a viscosity subsolution of 
()16p on 'D{V) if for any 93 G r and any 7]m G 'T>{V) such that v — ip has a || • ||-local maximum 
at i]M we have 

pv{riM) < {7]M, A*Vip{r]M)) + f{VM)^voiVM) + ^^2(%) + niipnoiVM))- 

(ii) A continuous function v : 'D{V) ^ M is called a viscosity supersolution of ()16p on ^{V) 
if for any 93 G r and any r^.^ G 'T>{V) such that v — if has a || • ||-local minimum at i]m we have 

pv{7]rn) > (??m, A*Vipir]rn)) + f (Vm) ^7jo i^rn) + C^2(?/o) + ■^(v^yyo (^"i))- 

(iii) A continuous function v : I'(V^) ^ M is called a viscosity supersolution of ()16p on I'(V^) 
if it is both a viscosity sub and supersolution. 
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We can prove the following: 

Theorem 4.3. The value function V is a viscosity solution of 1^6]) on T){V). 

Proof, (i) We prove that F is a viscosity subsolution. Let (r/jv/, ip) € T>{y) x r be such that 
V — if has a local maximum at rjM- Without loss of generality we can suppose V{rjM) = ^p{r]M)- 
Let us suppose, by contradiction that there exists > such that 

2i/ < pV{r]M) - {{r]M,A*\/ip{r]M)) + /(r/M)v'r,o (^m) + C^2(%) + W(</'»7o(^m))) ■ 

Let us define the function 

ip(r]) := V{r]M) + {Vip{r]M),V - Vm) + \\v - Vm\\-i- 

We have 

Thus is a test function and we must have also 

2u < pV{r]M) - {{r]M,A*\/ip{r]M)) + f{VM)^rio{VM) + C^2(?7o) +'H{ipno{r]M))) ■ 
By concavity of V we have 

V{riM) = ipM, V>{r]) > V{r]) + ||?7 - tjmWIi, r? G V{V). 
By the continuity property of ip we can find e > such that 

< pViv) - {{v,A*Vp{ri)) + f{ri)if^,{ri) +U2{vo) +n{^r,o{v))), V e B{riM,e). 

Take a sequence (5„ > 0, (5„ ^ and, for any n, take a (5„-optimal control c„(-) G CadiVM)- Set 
:= r/M; c„(-)) and define 

in := inf{t > | - ??m|| = e} A L 

Of course t„ is well-defined and belongs to (0, 1]. Moreover, by continuity of trajectories, G 
B{7]M,£), for t G [0,t„). We distinguish two cases: 

limsuptn = 0, or limsupt„ > 0. 



In the first case we can write 



Sn > 







> 



e""' [Uiicnit)) + U2{XSm dt - {e-P'-V{X{tn)) - V{iim)) 

e-"' [Ui{cn{t)) + U2{X^m dt - {e-P'-{ip{X^{tn))) - ^{m)) + e"''*" - t/mI 



> 



-pt 



-pt 



Uiicnit)) + f/2W(t)) + dt + e-P'-\\X^itn) - milti 

U2{X^{t)) - pp{X%t)) + A*V^{X\t))) 



+ dt + e-P'-\\X^{tn) - mif-i 



> tniy + e-P'-\\X''{tn) 



VM\\-l, 
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thus it has to be 

\\X^{tn)-VM\tl^O. 

Let us show that this is impossible. The above convergence impUes in particular that 

l^Eitn) - iVM)o\ ^ 0. (18) 

Moreover, by definition of t„, it has to be 

\XS{t)-ir]M)o\<e, te[0,tn]. (19) 
Since tn 0, taking into account (fT9l) . we have also 

\\X^{tn)-{VM)l\\Ll^^O. (20) 

The convergences (llSp and (I20p are not compatible with the definition of t„ and the contradiction 
arises. In the second case we can suppose, eventually passing to a subsequence, that tn —* i ^ 
(0, 1]. So we get as before 

Sn > + e-P'-\\X'^{tn) " r?Af > tnV' 

since (5„ — > and tnV tv, again a contradiction arises. 

(ii) The proof that V is a viscosity supersolution is standard, see e.g. [26]. □ 

4.2 Smoothness of viscosity solutions 

In this subsection we show that the concave \\ ■ ||_i-continuous viscosity solutions of (fTBl) (so 
that in particular the value function V) are differentiable along the direction h = (1, 0). For this 
purpose we need the following lemma. 

Lemma 4.4. Letv : 'D{V) — > M a concave \\-\\^i- continuous function and suppose thatfj G 1^{V) 
is a differentiability point for v and that \/v{fj) = ^. Then 

1. There exists a test function (p such that v — tp has a local maximum at f) and \7ip{fi) = C- 

2. There exists a test function ip such that v — p has a local minimum at f) and \/(p{fj) = 

Proof. Thanks to Proposition 13.121 and due to the concavity of v, the first statement is 
clearly satisfied by the function (•,C)- We prove now the second statement, which is more 
delicate. We use the notation of Proposition I3.12[ Thanks to the third claim of Proposition 
[37l2\ we have A*C € D+u{f)'). This means that 

u{r]') - u{f)') - (?/ - f)',A*0 > -\\r]' - f]'\\ ■ e{\\r]' - 7?'||), 

where e : [0, +oo) [0,+oo) is an increasing function such that ^(11?/ — ??'|[) — > 0, when 
II V ~ ^'11 ~^ 0- The previous inequality can be rewritten also as 

n(r/') - n(f?') - (A(r/' - f/'),C) > -\\v' " ^'11 • ^(11^' " Hi)- 

Passing to v this reads as 

v{r]) - v{t]) - (r? - r?, C) > - vW-i ■ e (||r/ - f/||_i) , 



20 



where e {\\ri — 0, when \\ri — 0. We look for a test function of this form: 

viv) = v{fj) + iv -v,0 + giWv - v\\~i) , 

where g : [0, +oo) — > [0, +00) is a suitable increasing function such that g{0) = 0. Notice 
that (f^fj) = v{fi), so that, in order to prove that v — tp has a local minimum at fj, we have to 
prove that < t> in a neighborhood of 77. 
Let us define the function 

/•2r j-2r 

g{r) := / e(s)ds > / e{s)ds > re{r). 

Jo Jr 

Then 

ip(r]) = v{fi+{'n-fj,C)-g{\\r]-fi\\_i) 

< v{fi) + {v-'n,C) - II?? - ??IU-i • £{\\ri - vW-i) 

< v{rj). 



Moreover 



V</^(r?) 



Wv - m\-i 



C, if r/ = r?, 

i.e. if \s a. test function and V(p{ff) = C,. □ 

Now we can state and prove the main result. 

Theorem 4.5. Let v he a concave \\ ■ \\^i-continuous viscosity solution of U6\} on ^{V). Then v 
is differentiahle along the direction h = (1, 0) at any point rj G 'T>{V) and the function ij Vrjoiv) 
is continuous on 'D{V). 

Proof. Let rj £ 1^(1^) and C)C ^ D*v{ff). Thanks to Proposition 13.121 there exist sequences 
(??n), {fjn) such that: 

• Vv{rjn) and Vv{fin) exist for all n G N; 

• A*Vv{y]n) A*C and A*Vv{fin) ^ A*^. 
Thanks to Lemma 14.41 we can write, for any n € N, 

pv{r]n) = {Vn,A*Vv{rin))+f{nn)Vr,o{Vn) + U2{Vo)+'H{Vrjo{Vn)), 
Pv{Vn) = {fln,A*Vv{rin)) + f {fln)Vr,o{fjn) + U2{ril) + HiVnoifin)) ■ 

Passing to the limit we get 

(7?, ^*C) + /(r/)Co + f/2(%) + W(Co) = pv{r]) = iv, A*0 + 1^0 + f^2(r/o) + ^(^0)- (21) 

On the other hand AC + (1 — A)^ € D~^v{ff)^ for any A S (0, 1), so that we have the subsolution 
inequality 

pv{^) < (,?,^*[AC+(l-A)e])+/(r?)[ACo+(l-A)^o]+i72(r?o)+H(ACo+(l-A)6), VAe (0,1). (22) 
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Combining ([2lD and (l22|) we get 



n{XCo + (1 - A)^o) > A?^(Co) + (1 - A)H(eo); 

since TC is strictly convex, the previous inequality implies Co = Co- This means that the projection 
of D*v{i]) onto n is a singleton. Thanks to (jlSp this implies also that the projection of D^v{i]) 
onto n is a singleton and therefore that v is differentiable in the direction n at r]. 

We prove now that the map r] i— > Vrjgir]) is continuous on !){]/). Let rj G T^{V) and let (r/") 
be a sequence such that r/" ^ ?]. We have to show that Wr;o('?") ^ ^f?o(^)- Of course for any 
n G N there exists € L^j. such that {vr^g{T]'^),Pi) G D^v{t]^). Since w is concave, it is also 
locally Lipschitz continuous so that the super-differential is locally bounded. Therefore, from any 
subsequence (v^q (r/"*^ ) ) > we can extract a sub-subsequence (u^q (r/"'='> ) ) such that (r/"'''> ) , ^ 
is weakly convergent towards some limit point. Due to the concavity of v this limit point must 
live in the set D~^v{r]). In particular the limit point of {v-qoi'n^*''^)) must coincide with v,^Q{r]). 
This holds true for any subsequence (t'Tyo (^"'°))' ^° ^^^^ claim follows by the usual argument 
on subsequences. □ 

Remark 4.6. Notice that in the assumptions of Theorem 14.51 we do not require that v is the 
value function, but only that it is a concave || • ||_i-continuous viscosity solution of (I16p . ■ 

Remark 4.7. Thanks to the regularity result of the previous subsection we can define the 
feedback map 

C(r?) := argmax^>o (C/i(c) - cVr,^{ri)) , rj G V{V), 

which, at least formally, should define an optimal strategy for the problem. 

The object of the forthcoming paper [20] is the study of the closed loop equation associated 
to this map and the proof of a Verification Theorem showing that this map actually defines an 
optimal feedback strategy for the problem. ■ 

Remark 4.8. When the delay is concentrated in a point in a linear way, we could tempted 
to insert the delay term in the infinitesimal generator A and try to work as done in Section [3l 
Unfortunately this is not possible. Indeed consider this simple case: 

y'{t) = ry{t) + y{t-T), 
y(0) = y{s) = rji{s), s G [-T,0), 

In this case we can define 

A : V{A) CH^H, {vo,mi-)) ^ (^^o + Vi{-T) , v'i{-)) ■ 

where again 

V{A) ■.= {r,eH\ 77i(-) G W''\[-T,0];R), 7yi(0) = m}. 
The inverse of A is the operator 

A-' : {H,\\ ■ II) iViA),\\ . II) (7?o,r/i(-)) ^ c + J^^miOd^^ , 

where 

1 r 

r + 1 r + 1 J„y 
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In this case we would have the first part of Lemma 13.71 but not the second part, because it 
is not possible to control |ryo| by Indeed take for example r such that = ^, and 

{ri"')n£N C H such that 



r?J = l/2, / <m = l, ne 



N. 



We would have c = 1/2, so that 



r 



0. Moreover we can choose 77" such that, when n — > 00, 



+ 



ds 



0. 



Therefore we would have Ir^g | = 1/2 and ||ry"||_i — > 0. This shows that the second part of Lemma 
l3.7l does not hold. Once this part does not hold, then everything in the following argument breaks 
down. ■ 
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